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Executive Summary 

A comparative study was conducted to compare two approaches to engineering design 
curriculum between different schools (inter-school) and between two curricular approaches, 
Project Lead the Way (PLTW) and Engineering Projects in Community Service (EPIC High) 
(inter-curricular). The researchers collected curriculum materials, including handouts, lesson 
plans, guides, presentation files, design descriptions, problem statements, and support guides. 

The researchers conducted observations in the classrooms to collect qualitative indicators of 
engineering/technology reasoning, collect data on the nature of students’ questions, how students 
define problems, and operate within the constraints of a design problem. 

Observational studies were conducted with students participating in Project Lead the 
Way and with students participating in Engineering Projects in Community Service (EPICS). 
Study participants were asked to work through an ill-defined problem, in this case the problem of 
creating a new playground for an elementary school. The data from these protocols were 
analyzed using a coding process; a list of universal technical mental processes (Halfin, 1973) and 
a computer program OPTEMP (Hill, 1997) to record frequency and time of each mental process 
employed by the students. The data were used to identify common cognitive strategies employed 
by the students and to determine where students placed greatest emphasis during the observation 
period. 


General findings indicated that participants in the EPICS-High program were in general 
more solution-driven problem solvers, while the Project Lead the Way participants were 
generally problem-driven as defined by Kruger & Cross (2006). Although the participants in 
both groups had completed advanced courses in mathematics; mathematics was rarely employed 
(less than 3%) to describe constraints of the problem or predict results of proposed solutions. 
Over half of the students became fixated at some point on the provided picture. (Smith, Ward, & 
Schumacher, 1993). This study provides important insight about how students solve ill-defined 
problems, providing vital information for technology education as it seeks to implement 
engineering design. 

This research was conducted through support of National Science Eoundation under Grant No. 

ESI-0426421 
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Introduction 

The purpose of this study was to better understand the current status of engineering-focused 
curriculum programs at the high school level and its greater impact on student learning. 
Specifically, this study examined the implementation of Project Lead The Way and EPICS High, 
engineering- focused curriculum and its impact on students’ abilities to employ the engineering 
design processes using mathematical and scientific reasoning to solve engineering design 
problems; a construct of interest to K-12 engineering education and the greater STEM 
community. 

The rise in engagement of secondary engineering education (Douglas, Iversen, & 

Kalyandurg, 2004) and the increase in development of engineering-focused curriculum for 
grades 9-12 (Dearing & Daugherty, 2004) provide strong rationale for examining pre-existing 
secondary engineering-focused programs to evaluate the impact on student learning. 

Background 

No Child Eeft Behind (NCEB) legislation generated a new age of accountability by placing 
emphasis on improving students’ achievement in the core subjects. Recent trends in secondary 
education have generated multiple efforts to integrate core subjects such as math and science 
with technology education. Researchers in technology education have investigated the effects of 
technology education instruction on performance in mathematics and science for over a decade 
(Childress, 1996; Merrill, 2001). Research indicates that engineering drawing and design courses 
have had a significant impact on student achievement on mathematics standardized end-of-year 
tests (Dyer, Reed, & Berry, 2006). Another study discovered that engineering design instruction 
at the middle school level reduced the performance gaps among certain ethnic groups and 
increase student’s conceptual knowledge of science, while simultaneously generating higher 
order thinking skills such as analysis and synthesis (Cantrell, Pekcan, Itani, &Velasquez-Bryant, 
2006). Each of these studies indicate that there is a growing interest in investigating how 
engineering-focused instruction impacts student learning. 

In addition, several initiatives have started in the last decade to introduce engineering 
principles and engineering reasoning into grade 9-12 classrooms (Brophy, Klein, Portsmore, & 
Rogers, 2000). While PETW and other curriculum programs like EPICS-High focus on different 
aspects, from engineering problem solving to service learning; each of these new curriculums 
share the lack of a cohesive body of systematic and rigorous research studies investigating the 
effectiveness of their programmatic efforts. Specifically, there is a lack of investigation on higher 
order thinking, open-ended problem solving and the development of design and engineering 
knowledge/skills (Chaker, 2008). 

Dyer, Reed, and Berry (2006) recommended more research on the instruction of 
engineering design to accurately determine what elements of instruction are most beneficial to 
student’s learning of math and science. Similarly, Sheppard, Pellegrino, & Olds (2008) and 
Brophy (2008) call for more research on engineering efforts in high schools. This becomes 
especially pressing, since individual states develop specific engineering standards (i.e. 
Massachusetts) and specific teacher licensures will likely follow. As these programs continue to 
grow, there is a need to build a strong base of rigorous research to provide educated and specific 
feedback on how to improve existing curricula and build a cohesive research agenda on 
engineering reasoning development in the K-12 grade spectrum. 

Participants 
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Project Lead The Way (PLTW) is a national pre-engineering program that has been 
implemented into a number of high school and middle schools. The Project Lead The Way 
program launched in 1997 based on previous work in the 1980s by Richard Blais at 
Shenendehowa Central School (Blais & Adelson, 1998). Project Lead The Way has grown to 
serve over 1300 schools and 175,000 students (Bottoms & Anthony, 2005). Over 280 high 
schools in the state of Indiana alone are currently implementing Project Lead The Way. Two 
PLTW schools in northwestern Indiana were selected for this study because they had 
successfully implemented PLTW at the high school level and had similar student demographics. 

Engineering Projects in Community Service (EPICS) is a national engineering-centered, 
academic service-learning program initiated at Purdue University (Coyle, Jamieson, & Oakes, 
2005). The main tenets of service learning are 1) curricular connections; 2) reflection, 3) 
community partnerships, 4) authentic, complex and ill- structured problems, 5) addressing real 
needs, and 6) performance-based assessment (Honnet & Poulsen, 1989). Multidisciplinary teams 
are formed with local not-for-profit organizations to define, design, build, test, deploy, and 
support engineering-centered projects that significantly improve the organization’s ability to 
serve the community. EPICS-High (for grades 9-12) began in 2007, with the aim of bringing the 
engineering design concept into the high school environment. The program has quickly grown to 
32 active high schools across the nation with over 650 students served, 50% female students, 
48% minority students, and over 50% in free and reduced lunch programs. Two EPICS-High 
programs in northwestern Indiana were chosen for their record of success in implementing the 
EPICS-High program. Every effort was made to select participating schools that align with the 
student body demographics. 


Table 1. Participating School General Student Demographics 


PLTW 1 

PLTW 2 

EPICS 1 

EPICS 2 

Enrollment 

883 

1606 

1833 

883 

Graduation Rate 

75.7% 

92.2% 

83.8% 

75.7% 

White 

72% 

88% 

86% 

72% 

Hispanic 

25% 

5% 

7% 

25% 

Multicultural 

2% 

3% 

2% 

2% 

Asian 

1% 

2% 

1% 

1% 

Black 

1% 

2% 

3% 

1% 

Native American 

1% 

0% 

1% 

1% 

Surrounding Area 

Rural 

Urban Fringe 

Urban Fringe 

Rural 
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Research Questions 

This study was guided by the following research questions. Since the nature of our research 
is comparative, the responses to each of the questions will be compared across different schools 
(inter-school) and across curricula (inter-curricula): 

1 . What are important elements embedded within an appropriate engineering design 
problem for high school students to study/solve? 

2. What elements in the engineering design problems (or in the curriculum unit) 
encourage teachers to engage mathematical and science curricula elements and support 
mathematical reasoning or scientific reasoning of their students? 

3. What are the most common elements within student dialogues as they define 
engineering, engage in student collaboration and class discussions when seeking to solve 
engineering design problems? Which attributes or elements of engineering are missing or 
strongly represented? 

4. What are common cognitive and meta-cognitive strategies employed by high school 
students as they work to solve an engineering design focused problem? 

Methodology 

The research team collected curriculum materials from the participating classrooms, 
including textbooks, handouts, lesson plans. Power Point files, design descriptions, problem 
statements, and support guides. The researchers conducted observations in the classrooms to 
obtain qualitative indicators of engineering/technology reasoning and to collect data on the 
nature of students’ questions, how students define problems, and how they operate within the 
constraints of an engineering design problem. 

Next, a protocol analysis session was conducted with a group of three student volunteers 
from each site. Each volunteer was given the same design problem (the transfer problem) shown 
in Appendix B. Each student was asked how they would proceed from the given problem 
statement in order to improve the current condition described in the statement. See Appendix A 
for the complete protocol. The students were asked to define the problem, list all constraints that 
they impose on the problem, and describe how they would proceed to solve the problem. The 
participants were asked to verbalize their thoughts as they worked through the ill-defined 
problem (verbal protocol methodology). The researchers prompted participants to keep talking 
through the problem. The testing sessions were limited to thirty minutes, but most lasted between 
ten and fifteen minutes. Certainly, in the time constraints imposed by the testing session, a 
student would be unlikely to reach the final stages of the design process. However, one of the 
most important stages of the engineering design process, framing the problem, occurs early in the 
process of solving a technical problem. Experts in the field of design report that framing the 
problem is a critical step to the design process which occurs as soon as the designer is presented 
with a technical problem (Dym, Agogino, Eris, Erey, & Eeifer, 2005; SchOn, 1983). The transfer 
problem was developed to share characteristics of an ill-defined, complex, and dynamic design 
problem (Jonassen, 2000). 

Analysis 

The project involves two separate analyses of artifacts, (1) curriculum materials, and (2) 
results from selected students transfer problem analysis. Both forms of data were analyzed with 
an analytical induction framework (Bogdan & Biklen, 1992) similar to the constant comparative 
method of grounded theory (Strauss & Corbin, 1990), combining data analysis, further literature 


6 



review, and theory-building in a cyclical manner. In the analytical induction approach, data built 
the basis for further descriptions and interpretations, and in contrast to grounded theory is 
informed by prior research. 

As students progressed through the transfer problem session, the students’ cognitive 
processes were identified and coded from a list of 17 universal mental processes (Halfin (1973). 
A computer analysis tool called the Observation Procedure for Technology Education Mental 
Processes (OPTEMP) (Hill, 1997) was utilized to capture, record, and organize the codes from 
each transfer problem session. The researchers coded the actions and cognitive processes used by 
each student participant as he or she worked through the engineering design problem. 

Triangulation 

Ary, Jacobs, Razavieh, and Sroensen (2006) indicate that collecting data from different 
sources and comparing these data sources is a good way to determine validity within a study. 

This comparison of multiple sources is called triangulation. Triangulation is a generally accepted 
research method to ensure validity. As multiple sources point to the same conclusions, the 
researcher can be relatively confidence that the researcher’s instruments are measuring what they 
are intended to measure appropriately. Eor the purpose of this study, triangulation was achieved 
through the use of: classroom observation, transfer problem results, review of curriculum 
material, and student / teacher surveys. Through multiple sources of data, the researchers were 
able to achieve validation by triangulating these findings. 

Limitations 

The researchers acknowledge that the study has a small n of approximately 60 students 
observed in PETW and EPICS High classes and an n of 12 participants for the observational 
protocol, therefore, the results of the study are not generalizable to all secondary engineering 
design programs. 

Other possible limitations include a potential bias regarding PETW and EPICS High. 
According to Merriam (1998) researchers must acknowledge potential biases within the final 
report of the study. The researchers acknowledge that a potential bias could exist regarding both 
PETW and EPICS high due to the fact that the researchers work within an engineering/ 
technology teacher preparation program that provides undergraduate students with certification 
in PETW courses and the fact that EPICS High was created by Purdue engineering education 
faculty. To help ensure that such biases did not taint research findings, the researchers 
implemented methods such as Merriam’ s observational elements guide, see appendix C. The 
observational elements guide provided researcher with guiding questions that required reflection 
on a variety of factors in order to maintain an objective perspective as the research collected 
observational data. The researchers also use triangulation methods to provide multiple 
perspectives and multiple forms of data. Einally, researchers used multiple raters on the 
observational protocol transfer problem session to maintain consistent and reliable data 
collection. These are sound research methodologies that can help to ensure an objective data 
collection process. 
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Findings 

Classroom Observations: 

A total of three class observations were conducted at each PLTW and EPICS-High site. Each 
PETW and EPICS High class session was 90 minutes for a total of 270 minutes of observation at 
each site. 

Project Lead the Way Observations 

The course observed was PETW Principles of Engineering. The PETW students were 
working on marble sorting activity during the observation sessions, the activity required student 
teams to design and build a device to sort marbles by color using Eishertechnik parts. An 
observational template was used (Merriam, 2001); see Appendix C. The nature of much of the 
classroom dialogs were regarding Eishertechnik parts (motors, sensors, structures, etc) and how 
the students were planning to connect the various parts to make the device function properly. 
However, as the team devices began to be tested, malfunctions often occurred. As a result of 
these device malfunctions; students worked as cohesive teams to troubleshoot various problems 
encountered. Student dialogs were healthy problem solving conversations and represented 
cooperative team efforts. However, students did not appear to be using a systematic approach to 
solve problems. The students approach to solving problems was reactive as they employed trial 
and error approaches, which might also be described as “tinkering”. Students often were 
observed reconfiguring the device parts such as wires, motors, and sensors. Another example of 
the PETW students limited systematic approach to solving problems through the design process 
was observed when one student voiced his concern to his instructor about being ill-prepared for 
the transfer problem session because he could not remember the order of the 12-steps 
engineering design process. Although, there is much more to engaging in the engineering design 
process then learning a list of steps within a design process, this observation may indicate the 
student’s lacked an understanding in how to use the engineering design process as a systematic 
approach to solving problems. 

EPICS High Observations 

Both EPICS High participating schools were observed for a total of three class observations. 
One important observation about both EPICS High classes was the use of an engineer’s 
notebook. Both Epics High programs required students to obtain and keep an engineer’s 
notebook for the course. The EPICS High School #1 class was discussing “great teams” during 
one of the classroom observations and “community” during another class observation. The 
community discussion was an EPICS High created lesson titled Your community. . .My 
community. The instructor led a class discuss by asking questions like “How big is your 
community?, Where does it begin and end?” The instructor encouraged students to think about 
how they individually define community, what did community mean to them. Eater in the class, 
the discussion led to the brainstorming possible community service projects to be explored by the 
class. One researcher had a conversation with the EPICS #I instructor who shared that his school 
started the EPICS High program to purposefully target the middle level academically performing 
students instead of targeting the high performing students. The course gave students an 
alternative to PETW which the school also offers. The Purdue EPICS program provided a 48,000 
dollar grant to provide the EPICS school #1 with new laptops for all the students to use in the 
class. The grant also allowed the school to travel to Aurora, IE to the Walter Payton Brew House 
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roundhouse. Last year’s EPICS High project was a feasibility study to restore the train 
roundhouse that is located in the community. EPICS High school #1 class is 12 weeks long. 

One classroom observation at EPICS High #2 school found small groups of students (two 
groups of two students) discussing with the instructor their EPICS High community projects. The 
instructor reminded the students that they needed to be recording their design thinking in the 
design notebooks and how the information was to be recorded. The instructor asked the students 
to present their design project ideas informally to the group. One group selected a community 
project to design and build a walking path for students on school grounds, the path would need to 
be wheelchair accessible. The design team was talking about making a pathway using some sort 
of gravel. The design discussion lead to brainstorming the different material they could use so 
the wheel chairs won’t get caught. The other design team selected a project to create a storage 
container that would allow teachers to use a magnetic key fob to lock and unlock their laptop 
computers in a storage unit. Many teachers at this school have complained about having to carry 
their laptops everywhere they go for security reasons instead of being able to just leave it in their 
classrooms or one central location. The EPICS High #2 class was small and as a result was a 
very informal classroom structure, however there was good classroom dialogue regarding the 
design process. The instructor emphasized the importance of creating a clear and concise 
problem statement and recording design work in the engineer’s notebook. The EPICS High #2 
instructor was new to the EPICS High program and had just completed EPICS training in the 
summer of 2009. The EPICS High #2 school program runs the EPICS High course as a club and 
not an official class; students do not get course credit and meet after school hours. 

Curriculum Documents 

A review of PETW curriculum documents for the course Principles of Engineering revealed 
a focus on teaching students about systems, subsystems, open and closed loop systems, basic 
computer programming, and troubleshooting problems within a system. Eater in the semester, 
students also learned about basic statics and dynamics. Eurthermore, students were taught how to 
use free body diagrams and conduct basic vector analysis. This knowledge would have been 
helpful for student who participated in the transfer problem. However, this instruction came after 
the transfer problem testing session. Upon review of the Principles of Engineering curriculum, 
the researchers identify that the students are exposed to a limited scope of the engineering design 
process. PETW curriculum program consists of a series of courses in pre-engineering; therefore 
it is unfair to fully assess students design capabilities upon their completion of just one course. 
Assessment of students who have experienced all the PETW courses would be ideal such as in 
the PETW capstone course: Engineering Design and Development (EDD). This course was 
studied in a similar protocol analysis study (Kelley, 2008). However, the schools in this study did 
not offer this course. Eurthermore, most of the students used in this study had taken Introduction 
to Engineering Design that provided a broader overview of the engineering design process so 
students should have been prepared to employ the engineering design process to solve the 
transfer problem. 

The EPIC High curriculum documents that were studied were lesson plan documents taken 
from EPICS High Curriculum Module 1- Design, week 3. The activities in this module focus on 
design notebooks. One activity required students to assess their current note-taking skills by 
examining their notebooks they keep for other classes. The activity asks students to brainstorm in 
a small group how the design notebook differs from traditional class notebooks. 
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Other similar class activities included making use of other people’s design notes, building 
note taking skills, and building organizing skills for design notebooks. Lessons for later in the 
course focused on the topic of solving engineering problems in teams. These lessons required 
students get into small or large design teams and work to solve some ill-defined problems. One 
problem was as follows: 

In your group, solve the following design problem: 

Landfill space is rapidly running out. Develop a plan to eliminate your city’s dependence 
on the local landfill. Imagine, for now, that the city’s population is 100,000. As you solve 
this problem, each member should take notes according to your design notebook 
guidelines.” (EPICS High curriculum Module 1- Design lesson plans, p.29). 

When students were finished exploring possible solutions, they were required to compile a 
design record providing a design rational including experimental measurements, sketches, safety 
precautions, design criterion rational. Students were also asked to consider assessing how they 
functioned as a design team during the assignment. This exercise in open-ending problem 
solving would provide ideal experiences for students and should prepare students for the transfer 
problem. See Appendix B. 


Ill-defined Transfer Problem Session: 

Table 2. Transfer Problem Participants’ Coursework 


Participant 

Grade 

level 

Math courses 

Science Courses 

Technology Courses 

PETW 1 

10 

A1,G 

ICP 

AutoCAD, POE 

PETW2 

10 

Al, A2 

Biology, Physics 

lED, POE 

PETW 3 

10 

Al, A2, G 

Bio, Chem 

lED, POE 

PETW 4 

10 

G, A-AP 


lED, POE 

PETW 5 

12 

G, A2, T, PC, C 


lED, POE, Bio-Tech 

PETW 6 

11 

G 1, G2, A3-A4, 
T,PC 

Bio, Chem, Phys 

lED, Design 

EPICS I 

10 

A1,G 

Bio, ICP 

EPICS, lED 

EPICS 2 

12 

Al 

Bio, Zoology, ICP 

EPICS, 

Manufacturing, 
Communications, 
Construction, 
Robotics, Machining 

EPICS 3 

12 

A2, G, PC, S 

Bio, Chem 

lED, EPICS 

EPICS 4 

12 

Al, A2, G, PC, 
C 

Bio, Chem, Phy, 
AP Phy 

lED, POE, EPICS 

EPICS 5 

12 

Al, A2, G, PC, 
C 

Bio, Chem, Phy, 
Chem2 

Manufacturing, 

Transportation 

EPICS 6 

12 

Al, A2, G, PC,C 

Bio, Chem, Phy, 
Chem2 

EPICS 


Math Courses Key: Al-Algebra; A2- Algebra II; G- Geometry; S- Statistics; T- trigonometry; 
PC- pre-calculus; C- calculus. Science Course Key: Ana- anatomy; Bio- Biology; Chem- 
Chemistry; ICP- Integrated chemistry and Physics; Phy- Physics. PLTW Course Key- lED- 
Introduction to Engineering Design; POE- Principles of Engineering 
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Project Lead The Way Transfer Problem Results 

There were several similarities in the Project Lead The Way participant groups when 
comparing their employed cognitive processes during the protocol session. Out of the seventeen 
cognitive processes that were first identified by Halfin (1973), the participants only used eight of 
the seventeen processes. See Table 3. Due to testing session constraints such as time limits, 
location, available resources it was expected that cognitive processes such as models/prototypes, 
measures, and testing as well as other cognitive processes would not be used when developing a 
design solution. On average the PLTW students used the analysis (AN) cognitive process 37.6 % 
of the time with a range of low 18% to a high of 61%. The design (DE) cognitive process was 
used 36.5% of the time with a low 21% and a high of 53% of the time. These were the two most 
employed cognitive processes by the PLTW participants. Besides the quantitative data that was 
collected, there were several similarities in the ways that the students went about solving the 
problem. 

Of the six PLTW students who were given this ill-defined problem, there was distinctive 
common pattern when comparing the two female participant results, see Ligure 1.1 & 1.2 and 
Table 4 & 5, participants #2 & #5 were females. Although the research design did not call for 
comparison of participants’ design thinking by gender, the frequency of the cognitive processes 
of the female students were similar, whereas those of the male students varied. Although these 
results are interesting, the researchers acknowledge that a greater sample size would be necessary 
to generalize the finding. 


Table 3. Halfin Code (1973) 


■ AN 

Analyzing 

■ CO 

Computing 

■ DE 

Designing 

■ OP 

Defining Problems 

■ 10 

Interpreting Data 

■ MA 

Managing 

MO 

Modeling 

■ PR 

Predicting Results 

QH 

Questioning/ hypothesis 
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Project Lead The Way 1 


IAN 
ICO 
I DE 
IDF 

no 

I MA 
MO 
I PR 
QH 





PLTW 1 


PLTW2 


PLTW 3 


Fig. 1.1: Project Lead Way Participating School #1 Time Breakdown 



Project Lead The Way 2 


PLTW 4 


PLTW 5 


PLTW 6 


■ AN 

■ CO 

■ DE 

■ DF 

■ ID 

■ MA 
MO 

■ PR 
QH 


MA PR 
(D 0% 2% 


Fig. 1.2: Project Lead Way Participating School #2 Time Breakdown 


EPICS High Transfer Problem Results 

With the exception of EPICS High participant #3, all other EPICS High students spent 1/3 or 
more of their time designing solutions with a high of 46 % time on (DE) Designing for EPICS 
participant # 6 to a low of 20% EPICS participant #3. On average, each EPICS High participant 
used seven of the nine Halfin coded cognitive strategies. In general the EPICS High students not 
only employed multiple cognitive strategies but also spent an ample percentage of time 
employing a variety of strategies. EPICS #1 and EPICS # 6 are the two female participants. See 
figures 1.3 & 1.4 and Tables 6 & 7. 
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Fig. 1.4: EPICS High Participating School #2 Time Breakdown 


PLTW results compared with EPICS High results 

The results of the EPICS High transfer problem observational protocol sessions reveal that in 
general the EPIC High students used a greater variety of cognitive capabilities when designing a 
solution to the transfer problem compared with PETW transfer problem results. Clearly, DE 
(Designing) and AN (Analysis) were the most often employed cognitive strategies for both 
groups, EPICS High participants employed a variety of other cognitive strategies for an ample 
amount of time, possibly indicating that the EPICS High students employed a more well-rounded 
approach to design by employing multiple cognitive strategies. Both groups, with the exception 
of one participant in each program (PETW #3 and EPICS #3), employed 1/3 or more of their 
time on the transfer problem in DE (Designing). This is a promising result because it indicates 
that the students were able to move successfully from the problem space to the solution space. 
However, 4 out of the 6 PETW participants spent 40% or more of their time on (AN) Analyzing. 
One EPICS High student spent 34% of time on (AN) Analyzing, all other EPICS High 
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participants dedicated less than 30% of their time using Analyzing as a cognitive strategy. It is 
important to note that the (AN) Analysis Halfin code was recorded by the researchers when 
participants were observed breaking down the problem and identifying constraints and criteria. 
(AN) Analysis and (DF) Defining Problems are cognitive strategies that represent the problem 
space of design thinking. Lawson (1979), identified problem solving strategies as either ‘problem 
focused’ or ‘solution focused’, and claimed the ‘solution focused’ strategies were more 
representative of a design-based problem solving strategy. 

Only PLTW participants #2 and #3 appeared to be stuck in the problem space, a common 
pattern for novice designers that limits their ability to generate solutions (Cross & Dorst, 1999; 
Dorst & Cross, 2001). The PLTW participants that placed emphasis on (AN) Analysis may have 
been performing this way because the PLTW students have had experience in identifying 
constraints and criteria in their PLTW course work and classroom activities. For example, the 
PLTW Principles of Engineering marble sorter activity (see description above in classroom 
observation section of the findings) requires the student teams to troubleshoot problems that 
occur in their design solutions by identifying the constraints and criteria that are limiting the 
success of the device. The results of this study indicate that these PLTW students may be 
transferring their knowledge and experience from the Principles of Engineering class to the 
transfer problem sessions. Although these results appear to identify a program-based impact on 
students’ ability to transfer their learning to an open-ended transfer problem, too much emphasis 
on (AN) Analysis can limit students’ abilities to move from problem space into solution space 
(Cross & Dorst, 1999; Dorst & Cross, 2001). 

Limited Mathematical Thinking 

Both EPICS High and PLTW students participating in the transfer problem observational 
protocol sessions had limited employment of (CO) Computing. The Halfin code computing is 
defined as “The process of selecting and applying mathematical symbols, operations, and 
processes to describe, estimate, calculate, quantity, relate, and/or evaluate in the real or abstract 
numerical sense” (Appendix B.) . The researchers did not expect that participants were going to 
be generating multiple mathematical models to develop solutions to this transfer problem in a 15 
to 30 minute test session; however, the transfer problem was created purposely with numbers 
embedded within the problem to see if students would attempt to quantify, estimate, calculate, or 
describe design solutions using the numerical information provided within the open-ended 
problem. None of the six participants spent more than 3% of their design thinking time using the 
(CO) Computing strategy. PLTW #4 spent the greatest amount of time computing, dedicating 
3% of time on this strategy, five other participants (PLTW #1, 3,6 and EPICS #1, 6) never 
employed computing once during the testing session. This finding reaffirms results in similar 
studies; for example, Kelley (2008) found that often the participants with the least amount of 
math instruction employed the most mathematical thinking. PETW # 4 had only two math 
courses (Geometry and AP Algebra) but applied the most mathematical thinking, 3% of the time. 
In contrast, PETW 5, PETW 6, EPICS 4, EPIC 5, and EPIC 6, each of whom had five or more 
math courses, actually employed (CO) Computing for 1% or less of the time. In fact, PETW 6 
had taken six math courses and EPICS 6 had taken five courses, yet neither employed any 
mathematical thinking during the protocol session. Some leaders in engineering design based 
instruction at the secondary education level suggest that mathematical modeling and 
mathematical analysis are key missing pieces in the technological design process (Hailey, et ah, 
2005; Wicklein, 2006). Although all student participants had taken advanced coursework in 
mathematics ranging from Algebra 1 through AP Calculus, the participants’ utilization of 
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mathematical strategies was minimal. This finding could cast doubt on the effectiveness of 
PLTW and EPICS High as effective strategies for improving the applicability of mathematics in 
engineering design activities. 

Design Example Fixation 

On the problem statement handout, there was a photo of a school under construction and its 
surrounding areas; see appendix B. The photo was used to provide a general context to the 
problem statement. All of the participants referenced the photo at some time in the testing 
session. Even though the photo was never referenced in the problem statement, all participants 
devoted attention to an analysis of the photo. Each participant at some point in the testing session 
used the photo like a template for the location of their playground. This discovery was 
unexpected; however. Smith, Ward, and Schumacher (1993) reported that providing a designer, 
whether novice and expert, with a design sample can cause the designer to become fixated on 
that design example. Such a design sample both limits and influences the designer. Kelley (2008) 
also found that participants in a similar protocol study were often fixated on the picture that was 
included in problem statement. Some participants in that study even asked questions about the 
picture and carefully studied URE citation for the graphic image. 

In this study, students were potentially limited in their creative thinking by the context of the 
photo of a school under construction. This can be an extremely important finding for K-12 
engineering educators because it illustrates the potential negative impact of providing an existing 
design example to students before they generate their own design ideas. There is some concern 
about the possibility that students may believe that solutions to assigned problems are contained 
within the student handout. 

General Observations across all groups (inter-curricula): 

Classroom Layout-The two Project Eead The Way classrooms were arranged differently than a 
traditional technology education classroom that contains workbenches, industrial machinery, and 
hand tool cabinets; the PETW classroom resembled a science lab. The classrooms had only 
“clean-work” equipment i.e. plotter, 3-D printer, scanners, computers. Although both schools 
had traditional unit shop facilities within the schools, these facilities were not used for the PETW 
classes we observed. EPICS High classes had similar layouts. EPICS High school #1 used a 
chemistry classroom for the EPICS High course and EPICS High school #2 used a PETW 
classroom that was similar to the description of the PETW classrooms above. 

Culture Shift- The general observations of the researchers identified a shift in classroom culture 
from traditional technology education courses. The researchers not only found the classroom 
layout and general characteristics of PETW and EPICS High students to be different from 
traditional technology education settings but also noted differences in student learning activities 
and in attitudes toward class activities. The students appear to take more pride in “minds-on” 
approaches to solving problems and in some cases displayed a lack of interest in hands-on work. 
This type of thinking could stem from having much of their activities based upon computer work 
and programming computer controlled machines, especially in the PETW courses. 
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Table 4. Project Lead The Way School #1 Transier Problem Results 
^ PLTW #1 r PLTW #2 ^ PLTW #3 


PLTW School 1 Mean 
Scores 


Halfin 

Code 

Freq. 

Time 

Freq. 

Time 

Freq. 

Time 

Freq. 

Time 

AN 

10 

01:38.4 

13 

05:48.5 

11 

07:40.3 

11.3 

05:02.4 

CO 

0 

00:00.0 

1 

00:08.4 

0 

00:00.0 

0.3 

00:02.8 

DE 

8 

03:00.7 

8 

03:14.2 

8 

02:39.7 

8.0 

02:58.2 

DF 

6 

02:00.0 

1 

00:44.1 

1 

01:02.8 

2.7 

01:15.6 

ID 

1 

00:03.8 

0 

00:00.0 

2 

00:11.6 

1.0 

00:05.1 

MA 

0 

00:00.0 

1 

00:02.8 

2 

00:03.4 

1.0 

00:02.1 

MO 

8 

02:12.4 

1 

00:13.6 

1 

00:31.1 

3.3 

00:59.0 

PR 

2 

00:09.5 

3 

00:25.1 

4 

00:21.4 

3.0 

00:18.7 

QH 

0 

00:00.0 

2 

00:15.0 

1 

00:01.5 

1.0 

00:05.5 


Table 5. Project Lead The Way School #2 Transfer Problem Results 
^ PLTW #4 r PLTW #5 ^ PLTW #6 


PLTW School 2 Mean 
Scores 


Halfin 

Code 

Freq. 

Time 

Freq. 

Time 

Freq. 

Time 

Freq. 

Time 

AN 

15 

02:56.2 

34 

11:17.1 

11 

04:19.3 

20.0 

06:10.9 

CO 

3 

00:27.9 

2 

00:09.6 

0 

00:00.0 

1.7 

00:12.5 

DE 

17 

06:47.2 

32 

08:06.5 

21 

09:07.5 

23.3 

08:00.4 

DF 

2 

01:21.1 

3 

00:59.2 

2 

00:58.6 

2.3 

01:06.3 

ID 

5 

00:38.6 

8 

00:51.8 

0 

00:00.0 

4.3 

00:30.1 

MA 

2 

00:09.0 

3 

00:21.8 

1 

00:12.7 

2.0 

00:14.5 

MO 

8 

02:08.7 

6 

00:50.6 

14 

02:00.3 

9.3 

01:39.9 

PR 

2 

00:05.6 

8 

00:38.9 

1 

00:03.9 

3.7 

00:16.1 

QH 

0 

00:00.0 

0 

00:00.0 

2 

00:32.6 

0.7 

00:10.9 




Table 6. EPICS High School #1 Transfer Problem Results 

^ EPICS #1 ^ EPICS #2 ^ EPICS #3 


EPICS School 1 Mean 
Scores 


Halfin 

Code 

Freq. 

Time 

Freq. 

Time 

Freq. 

Time 

Freq. 

Time 

AN 

4 

02:36.4 

9 

01:52.1 

14 

04:08.8 

9.0 

02:52.4 

CO 

0 

00:00.0 

1 

00:04.5 

1 

00:18.9 

0.7 

00:07.8 

DE 

5 

02:26.6 

14 

03:31.8 

14 

02:53.0 

11.0 

02:57.1 

DF 

1 

01:01.2 

4 

01:39.1 

7 

01:48.5 

4.0 

01:29.6 

ID 

0 

00:00.0 

2 

00:11.4 

6 

00:57.9 

2.7 

00:23.1 

MA 

1 

00:04.5 

2 

00:05.9 

1 

00:08.9 

1.3 

00:06.4 

MO 

3 

00:44.4 

3 

00:32.8 

12 

03:40.0 

6.0 

01:39.1 

PR 

0 

00:00.0 

4 

00:49.9 

1 

00:12.7 

1.7 

00:20.9 

QH 

4 

00:49.4 

1 

00:22.3 

0 

00:00.0 

1.7 

00:23.9 


Table 7. EPICS High School #2 Transfer Problem Results 

^ EPICS #4 ^ EPICS #5 ^ EPICS #6 


EPICS School 2 Mean 
Scores 


Halfin 

Code 

Freq. 

Time 

Freq. 

Time 

Freq. 

Time 

Freq. 

Time 

AN 

6 

00:40.9 

5 

00:45.4 

5 

01:21.6 

5.3 

00:56.0 

CO 

1 

00:03.5 

1 

00:04.4 

0 

00:00.0 

0.7 

00:02.6 

DE 

10 

02:34.4 

7 

01:57.8 

8 

02:40.8 

8.3 

02:24.3 

DF 

3 

01:20.3 

1 

00:49.4 

2 

01:07.1 

2.0 

01:05.6 

ID 

0 

00:00.0 

0 

00.00.0 

0 

00:00.0 

0.0 

00:00.0 

MA 

1 

00:04.7 

1 

00:13.3 

0 

00:00.0 

0.7 

00:06.0 

MO 

4 

00:32.8 

3 

00:27.0 

6 

01:13.7 

2.7 

00:44.5 

PR 

3 

00:15.1 

3 

00:13.0 

4 

00:06.0 

3.3 

00:11.4 

QH 

0 

00:00.0 

2 

00:13.8 

1 

00:36.4 

1.0 

00:16.7 




Inter-rater Reliability Results 

Two researchers were used to code the transfer problem observational protocol sessions. The 
two researchers were trained by the PI of this study how to interpret the Halfin code and how to 
record the codes using the OPTEMP software. The researchers practiced coding protocol 
sessions using sample videotape sessions are a part of their data analysis training. Both research 
assistants were present for all transfer problem sessions and each researcher coded independently 
using the OPTEM software. Pearson correlation coefficients were calculated to determine how 
reliable the observational results were for the two independent raters. The Pearson correlation 
coefficient is identified as a correlation coefficient which applies to interval and ratio level data. 
The Pearson R was calculated between the two raters for each transfer problem to determine how 
consistently the raters identified, timed, and reported student’s employment of the individual 
cognitive strategies. A Pearson R of 0 indicates no correlation, a Pearson R of -1 indicates a 
perfect inverse relationship, and a Pearson R of 1 indicates a perfect positive correlation. The 
researchers hoped for correlation close to 1, indicating that both raters agree and affirming the 
reliability of the results. The lowest Pearson correlations were .76 and .75 for PETW #1 and 
PETW #2, the first two transfer problem sessions that were coded by the research assistants. It is 
logical that the researchers were refining their coding abilities during these first two sessions. 
Subsequent correlations were .93 or higher, indicating a strong correlation and reliable coding of 
the transfer problem sessions. See Table 8. 

Table 8. Inter-rater Reliability- Pearson R results 


PETW participants 

Pearson R 

EPICS High Participants 

Pearson R 

PETW #1 

0.76 

EPICS High #I 

0.98 

PETW #2 

0.75 

EPICS High#2 

0.99 

PETW #3 

0.92 

EPICS High #3 

0.99 

PETW #4 

0.93 

EPICS High #4 

0.94 

PETW#5 

0.99 

EPICS High #5 

0.94 

PETW#6 

0.93 

EPICS High #6 

0.95 


Conclusions /Implications 

The researchers acknowledge that classroom observations were limited to approximately 60 
students observed in PETW and EPICS High classes and a sample of 12 participants for the 
observational protocol, therefore, the results of the study are not generalizable to secondary 
engineering design programs. The results of this study indicate that students from both 
engineering design programs successfully develop problem solving abilities to move from 
problem space to solution space as they worked through an open-ended ill-defined problem. 
Eurthermore, classroom observation revealed that student design teams had healthy design 
discussions and worked cooperatively to solve technical problems. 

However, students were limited in their ability to use mathematical thinking as a design tool 
to help create design solutions. Although most of the students had advanced math classes; over 
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50% had 3 or more classes and four participants had five math classes each, students spend 3% 
or less time using mathematics in their design thinking. This finding confirms prior findings in 
similar research. Kelley (2008), in a study of two approaches to design instruction that also used 
a protocol session with a transfer problem, found that mathematical thinking was a limited 
cognitive strategy employed by the protocol session participants. Moreover, Kelley and Wicklein 
in a national descriptive study of engineering design curriculum content found a low emphasis on 
mathematics and engineering sciences in current technology education courses (2009a). 
Technology education teachers also place a low emphasis on mathematics in student project 
assessment; the individual item using mathematics to optimize and predict design results yielded 
the lowest percentage of assessment time (Kelley & Wicklein, 2009b). It appears that technology 
education teachers are not placing great emphasis on using mathematical models to predict 
design results. Technology education teachers appear to recognize this limitation and have 
identified it as a challenge. Kelley and Wicklein (in press) in the national status study of 
technology education regarding engineering design also found technology education teachers 
identified: integrating the appropriate levels of mathematics and science into instructional 
content as a major challenge in implementing engineering design. More effort is needed to 
identify appropriate mathematical models and science inquiry examples that teachers can 
integrate into engineering design activities. EPICS High teacher #2 provided some insight on 
why students’ had limited use of mathematics in the transfer problem in response the question, 
“Do you believe that your students will employ mathematical thinking (when appropriate) as 
they work to solve the transfer problem?” 

Yes, I have given similar design problems that required students to use math to solve the 

problem. However, if students can see that they don’t need the math to begin designing 

then I think they won’t use math. . . .if they can find a way to solve the problem without 

math... they will.” 

These research findings should be considered by educators and curriculum developers of pre- 
engineering or secondary engineering design curriculum, of which many also promote these 
programs as ideal platforms for STEM education. These findings indicate that currently the 
transfer of STEM learning through the engineering design process on ill-defined problem solving 
is limited. Engineering design curriculum developers must be more purposeful in creating 
learning experiences that embed mathematical problems and science inquiry activities into the 
engineering design process. More efforts need to take place to locate appropriate ill-defined 
problems that can be explored through the engineering design process that will authentically 
engage students in the analysis and optimization stages of the engineering design process. 
Eurthermore, greater efforts must take place across the nation to provide professional 
development opportunities to assist math, science, and technology education teachers to locate 
appropriate levels of STEM subjects for delivery in pre-engineering or secondary engineering 
design curriculum. 

Future Studies 

This study has provided insight into approaches to teaching engineering design content at the 
secondary level. Additional research could strengthen the understanding of student learning 
when engineering design challenges are introduced into secondary classrooms. Consequently, 
these recommendations for further studies seem appropriate: 

(a). Earger observational protocol studies - While insights have been obtained from this study 
extending prior work (Kelley, 2008), the sample size of 12 is still quite small. There is a need for 
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observational protocol studies of larger samples of students who are engaged in engineering 
design activities that incorporate authentic levels of math and science inquiry. 

(b) Studies of expert designers - Engineering education can learn from observational protocol 
studies of expert engineers and expert designers as they working through design problems. For 
example, observational studies of expert designers can provide insight into typical percentages of 
time devoted to the respective cognitive capabilities in the engineering design process. 

(c) Revisit Halfin’s study - A contemporary replication building upon the work of Halfin (1973), 
who conducted a Delphi study involving ten expert technical problem solvers, and upon the more 
recent work of Hill (1997) and of Wicklein and Rojewski (1999). Expert technical problem 
solvers today are expected to employ the engineering design process and to meet the challenges 
and constraints imposed by the integrated global setting (Friedman, 2005; Pink, 2005). 
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Appendix A. Test Session Instructions 

Project Title: Cognitive Processes of High School Students Solving Ill-defined Technical 

Problem s 

This project has been created to learn about how high school students solve ill-defined 
technical problems. 

This is to inform you about what to expect as you work with me on this research project 

today. 

• I will ask you to fill out a simple survey about the type of classes you have taken in 
high school. 

• I will observe you as you work to solve an ill-defined technical problem. 

• I will videotape you as you work through the ill-defined problem. The purpose of the 
videotape is to provide a backup record of the test session and only will be used to 
verify the coding process. No one other than a fellow researcher and I will view the 
videotape of your problem solving session and the tape will be erased when I 
complete my research. 

• I will also be taking notes and using a computer program while you are working to 
solve the ill-defined technical problem. The computer is used to record your 
thoughts and processes by using a special coding process that I enter using the 
keyboard. 

• Your participation in this project will not affect your grades in school. I will not use 
your name on any papers that I write about this project. 

• If you want to stop participating in this project, you are free to do so at any time. 

You can also choose not to answer any question. 

• Don't worry about getting the right answer; as with most design challenges there is 
more than one right way to approach a problem and develop a solution. 

• If you have any questions about this testing session or process, feel free to ask me. If 
you have questions as you work through the problem, feel free to express them; 
however, I may not be able to answer. I am looking for your thoughts to this 
problem. 

• I will encourage you to keep talking through your thought process as you work on 
the ill-defined problem. 

• You can take a break at any time if you need one. Just let me know if you need a 
break and we can pause the camera and computer program. 

• I encourage you to NOT talk with other classmates about this session until 
tomorrow. 

If you have any other questions at this time feel free to ask. Thank you for taking the time to 
help me on this research project. 


Sincerely 
Dr. Todd Kelley 
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Appendix B. Transfer Problem 


Problem 

A new K-5 elementary school has been constructed in the local area and new 
playground still needs to be designed. You obtained the following specifications: 

The school is expected to have around 500 students. The area of the playground has not 
been determined and space is limited. Safety of both students and school properties 
needs to be considered. For example, if students are playing softball too close to a 
building, they would risk breaking windows. At minimum, one entire grade will use the 
playground space at once. The playground needs to be attractive and fun for all 
students. 

Your Task 

Describe how you would design the playground for the school in the problem 
statement. Please describe all assumptions, information you need to obtain, and 
justification of use of space as you "think aloud" your strategies for developing the 
solution. 
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Appendix C. Merriam's Observational Element Guidelines 


Merriam (2001) suggests six elements to consider when making observations. The 
following elements will be used as a guide when making classroom observations for this 
research and have been modified to address specific issues in this construct. 

1. Physical setting: What is the technology classroom like? How is it decorated and 
arranged? Where is the classroom location in the school building? What objects and 
artifacts are in the classroom? What other resources such as computers, equipment, 
etc are present? 

2. The participants: Who is in the classroom? Who are the students who are taking the 
engineering design course? What are the commonalities of these individuals? What 
are the differences of these individuals? What demographic is underrepresented? 
Who may be overrepresented? 

3. Activities and interactions: What types of activities are taking place in the 
classroom? How do participants interact with the activity and with each other? What 
are the objectives for the class? How are these objectives achieved? How are these 
objectives achieved? (Note: some of these questions may be revealed through curriculum 
documents). 

4. Conversations: What is the content of the conversations in this classroom? Who is 
speaking? Who is silent? What is being said nonverbally? Direct quotes from the 
class will be used when possible. 

5. Subtle factors: Informal or unplanned events. Nonverbal communication. Dress and 
physical appearance of participants. What events do not occur that should occur? 
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Appendix D. The Cognitive Processes identified by Halfin's 1973 Dissertation Study 


The Original Cognitive Processes identified by Halfin's 1973 
Study of High-level Designers 

Proposed 
mental methods 


Definition 

Analyzing 

AN 

The process of identifying, isolating, taking 
apart, breaking down, or performing similar 
actions for the purpose of setting forth or 
clarifying the basic components of a 
phenomenon, problem, opportunity, object, 
system, or point of view. 

Communicating 

CM 

The process of conveying information (or ideas) 
from one source (sender) to another (receiver) 
through a media using various modes. (The 
modes may be oral, written, picture, symbols, 
or any combination of these.) 

Computing 

CO 

The process of selecting and applying 
mathematical symbols, operations, and 
processes to describe, estimate, calculate, 
quantity, relate, and/or evaluate in the real or 
abstract numerical sense. 

Creating 

CR 

The process of combining the basic components 
or ideas of phenomena, objects, events, 
systems, or points of view in a unique manner 
which will better satisfy a need, either for the 
individual or for the outside world. 

Defining problem(s) 

DF 

The process of stating or defining a problem 
which will enhance investigation leading to an 
optimal solution. It is transforming one state of 
affairs to another desired state. 

Designing 

DE 

The process of conceiving, creating inventing, 
contriving, sketching, or planning by which 
some practical ends may be effected, or 
proposing a goal to meet the societal needs, 
desires, problems, or opportunities to do things 
better. Design is a cyclic or iterative process of 
continuous refinement or improvement. 
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Experimenting 

EX 

The process of determining the effects of 
something previously untried in order to test 
the validity of an hypothesis, to demonstrate a 
known (or unknown) truth or to try out various 
factors relating to a particular phenomenon 
problem, opportunity element, object, event, 
system, or point of view. 

Interpreting data 

ID 

The process of clarifying, evaluating, explaining, 
and translating to provide (or communicate) 
the meaning of particular data. 

Managing 

MA 

The process of planning, organizing, directing, 
coordinating, and controlling the inputs and 
outputs of the system. 

Measuring 

ME 

The process of describing characteristics (by the 
use of numbers) of a phenomenon problem, 
opportunity, element, object, event, system, or 
point of view in terms which are transferable. 
Measurements are made by direct or indirect 
means, are on relative or absolute scales, and 
are continuous or discontinuous. 

Modeling 

MO 

The process of producing or reducing an act, or 
condition to a generalized construct which may 
be presented graphically in the form of a 
sketch, diagram, or equation; presented 
physically in the form of a scale model or 
prototype; or described in the form of a written 
generalization. 

Models/prototypes 

MP 

The process of forming, making, building, 
fabricating, creating, or combining parts to 
produce a scale model or prototype. 

Observing 

OB 

The process of interacting with the 
environment through one or more of the senses 
(seeing, hearing, touching, smelling, tasting). 

The senses are utilized to determine the 
characteristics of a phenomenon, problem, 
opportunity, element, object, event, system, or 
point of view. The observer's experiences, 
values, and associations may influence the 
results. 
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Predicting 

PR 

The process of prophesying or foretelling 
something in advance, anticipating the future 
on the basis of special knowledge. 

Questions/hypotheses 

QH 

Questioning is the process of asking, 
interrogating, challenging, or seeking answers 
related to a phenomenon, problem, 
opportunity element, object, event, system, or 
point of view. 

Testing 

TE 

The process of determining the workability of a 
model, component, system, product, or point of 
view in a real or simulated environment to 
obtain information for clarifying or modifying 
design specifications. 

Visualizing 

VI 

The process of perceiving a phenomenon, 
problem, opportunity, element, object, event, 
or system in the form of a mental image based 
on the experience of the perceiver. It includes 
an exercise of all the senses in establishing a 
valid mental analogy for the phenomena 
involved in a problem or opportunity. 
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Appendix E. Research Poster 
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Appendix F. Teacher Follow-up Questions 
Good Morning, 

I have completed the final report for the PLTW/ EPICS High study and would like to share the 
results with you, but before I do I have a few follow-up questions that I believe would only take 
about five minutes to complete. Feel free to answer them within this e-mail. 

How do you think your students did on the transfer problem? The results revealed that 
there were two general patterns of design thinking, students in general were either Problem 
Focused or Solution Focused. Problem Focused is when students focus a majority of their time 
defining the problem and identifying constraints and criteria. Solution Focused is when students 
focus on brainstorming ideas and sketching. Based upon the assumption that all of your 
students performed similarly: 

1. Do you think that your students who participated in the transfer problem were Problem 
Focused or Solution Focused? 


2. Based upon your response to question #1, do you believe that students solved the 
transfer problem in a similar way to which they might solve a problem in your 
class/curriculum? 


3. Do you believe that in general, students use the design process as a universal problem 
solving process? 


4. Do you believe that your students will employ mathematical thinking (when appropriate) 
as they work to solve the transfer problem? 


5. Do you believe, based upon your students experiences in your class (PLTW or EPICS) that 
the students are prepared to engage in solving open-ended problems using STEM 
(Science, Technology, Engineering, Mathematics) thinking? 
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Appendix G. Teacher Follow-up Responses. 

PLTW # 1 Teacher Follow-up Response. 

1. Do you think that your students who participated in the transfer probiem were Probiem Focused or 
Soiution Focused? 

I think that my students were solution focused. They generally tried to create a solution, then go 
back to see if it fits within the constraints. 

2. Based upon your response to question #1, do you beiieve that students might soive the transfer 
probiem in a simiiar way to which they might soive a probiem in your ciass/curricuium? 

Yes, I believe that the students would approach problems in my classroom the same way they 
approached the transfer problem. Their approach to a problem, in my observation, tends to be 
habitual, and tied into their personality as much as anything. 

3. Do you believe that in general, students use the design process as a universal problem solving 
process? 

I do not believe that students generally approach problem solving using the problem solving 
process taught in PLTW. Some students formulate solutions and use guess and check to see if 
they fit the constraints. Some start with a concept of a solution, and fill in details that make it fit 
the constraints. They are prone; in particular, to immediately start judging solutions during the 
brainstorming phase. 

4. Do you believe that your students will employ mathematical thinking (when appropriate) as they 
work to solve the transfer problem? 

Again, I believe this varies with each student. I think this may dependent on learning style. 
Students who are visual learners may try focusing more on prototyping and sketching to figure 
solutions. Students who are more abstract in their thinking may use a more mathematical 
approach 

5. Do you believe, based upon your students experiences in your class (PLTW or EPICS) that the 
students are prepared to engage in solving open-ended problems using STEM (Science, Technology, 
Engineering, Mathematics) thinking? 

Of the students in my PLTW class, I believe most of them are prepared to approach open-ended 
problems in the STEM arena. My experience with PLTW is that students get out of it what they put 
into it. Students who do not put forth much effort in the class will probably not be as prepared to 
engage in these problems as students who were more attentive and conscientious throughout the 
course. 

PLTW #2 Teacher Response: 

1. Do you think that your students who participated in the transfer problem were Problem Focused or 
Solution Focused? 

I would assume that my students were solution focused. It seems that with a lot of the 
PLTW projects more emphasis is put on the final product or solution to the problem. Many of 
my students rush to sketch ideas out on paper to solve the problem. 

2. Based upon your response to question #1, do you believe that students might solve the transfer 
problem in a similar way to which they might solve a problem in your class/curriculum? Yes, as I stated 
in my previous answer about the PLTW curriculum seems to put more emphasis on the final 
product and not on the problem. 
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3. Do you believe that in general, students use the design process as a universal problem solving 
process? 

No, I believe most students use the scientific process to solving problems. Both processes 
have similar steps within each of them to help solve the problems. 

4. Do you believe that your students will employ mathematical thinking (when appropriate) as they work 
to solve the transfer problem? 

Yes, my students were taught at a young age and continue to learn that they need to follow 
a series of steps to solve math problems. By learning to solve problems this way, it probably 
carries over to solving the transfer problem as well. 

5. Do you believe, based upon your students experiences in your class (PLTW or EPICS) that the students 
are prepared to engage in solving open-ended problems using STEM (Science, Technology, Engineering, 
Mathematics) thinking? 

I would say yes, that most of my students are prepared to engage in solving open-ended 
problems using STEM, but in reality I know some of my students would still need extra help 
(ex. many questions to ask for clarification about the problem) in solving the problem. 


EPICS #1 Teacher Response: 

1. Do you think that your students who participated in the transfer problem were Problem Focused or 
Solution Focused? 

I would assume that the students were more problem focused 

2. Based upon your response to question #1, do you believe that students might solve the transfer 
problem in a similar way to which they might solve a problem in your class/curriculum? 

Yes, the students in class are often caught up in the problem and struggle to see solutions. 

3. Do you believe that in general, students use the design process as a universal problem solving 
process? 

No, most students in my class just try and try again, without focusing on a solution. 

4. Do you believe that your students will employ mathematical thinking (when appropriate) as they work 
to solve the transfer problem? 

No, the students are too methodical in how they approach problem solving. 

5. Do you believe, based upon your students experiences in your class (PLTW or EPICS) that the students 
are prepared to engage in solving open-ended problems using STEM (Science, Technology, Engineering, 
Mathematics) thinking? 

Yes, many can use these skills effectively if they take the time to evaluate. 

EPICS #2 Teacher Response: 

1. Do you think that your students who participated in the transfer problem were Problem Focused or 
Solution Focused? The students would be solution focused I think. 

2. Based upon your response to question #1, do you believe that students might solve the transfer 
problem in a similar way to which they might solve a problem in your class/curriculum? 

Yes, I would hope that the students would transfer their learning from my class. 
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3. Do you believe that in general, students use the design process as a universal problem solving 
process? Yes, in our area (technology education) but school wide no. 


4. Do you believe that your students will employ mathematical thinking (when appropriate) as they work 
to solve the transfer problem? 

Yes, I have given similar design problems that required students to use math to 
solve the problem. However, if students can see that they don’t need the math to 
begin designing then I think they won’t use math... .if they can find a way to solve 
the problem without math... they will. 


5. Do you believe, based upon your students experiences in your class (PLTW or EPICS) that the students 
are prepared to engage in solving open-ended problems using STEM (Science, Technology, Engineering, 
Mathematics) thinking? 

Yes, I believe that they have the ability to use STEM thinking. 
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